|
|
|
ABSTRACT |
Effective decision making and communication are essential skills for sports officials, who frequently report experiencing considerable stress across various sports. This study evaluated the impact of a stress management program on elite volleyball referees. The intervention aimed to reduce stress and anxiety while enhancing coping strategies. Thirty-eight referees (24 males, 14 females) participated in a randomized intervention guided by two experienced applied sport psychologists. Stress and physiological measures were assessed before and after officiating. Anxiety and officiating-related stress were evaluated using the State-Trait-Anxiety Inventory (STAI) and an adapted version of the Basketball Officials Source of Stress Survey (BOSSS-d), respectively. Cardiac responses, including heart rate (HR) and heart rate variability (HRV), were monitored during games. The program’s effectiveness was assessed using the Inventory of Quality Sport Psychological Support (QS17). Although the pre- and post-intervention comparisons did not reveal significant changes in anxiety, reported sources of officiating-related stress, or cardiac responses, findings from the QS17 indicated potential benefits of the stress management program for elite volleyball referees, highlighting avenues for sport psychological support and interventions. Future research, particularly longitudinal studies, is needed to further explore referees’ stress experiences during officiating. |
Key words:
Stress management, decision making, stress, officiating, sport psychology
|
Key
Points
- The current study examined the relationship between stress, anxiety, heart rate, and heart rate variability among elite German volleyball referees.
- An intervention program was devised with the goal of diminishing stress, anxiety, and negative affect among volleyball referees while also improving coping strategies.
- The results indicate that elite volleyball referees report minimal sources of stress and anxiety.
- The Inventory of Quality Sport Psychological Support ratings indicated enhanced cognitive appraisal and coping strategies following stress management program completion.
|
Stress and anxiety among elite volleyball referees while officiatingThe role of referees in sports is pivotal and complex, requiring rapid assessment of actions, prompt decision making, and effective management of matches involving highly emotional athletes and teams (Tuero et al., 2002). While considerable research has explored stress in high-performance athletes (Osório et al., 2017; Prapavessis et al., 1992; Tanguy et al., 2018), studies focusing on referees, especially volleyball referees, remain relatively scarce. This gap is significant, as critical factors, such as rule knowledge application, situational decision making, and effective communication, often shape referees’ experiences with potentially stressful events and merit simultaneous examination (Cunningham et al., 2018; MacMahon et al., 2014; Schrödter, Noël et al., 2023). Furthermore, critical game situations, such as tight scoring, objections from players or coaches, and challenging decisions, frequently trigger stress reactions, such as emotional responses like heightened anxiety among officials. These diverse sources of stress can lead to decreased performance and impaired decision making (Kostrna and Tenenbaum, 2021). Reduced performance by referees can not only affect the fairness and quality of the match but may also influence the success and outcomes for players and coaches, highlighting the broader implications of referee stress. Stress is a natural process wherein external stimuli (life events) are interpreted based on their perceived impact on performance and well-being, subsequently influencing an individual’s emotional and physiological state (Lazarus, 2000). According to Lazarus’ (2000) Transactional Model of Stress, stress emerges from the interaction between the individual and their environment, mediated through primary and secondary appraisals. Primary appraisal involves the subjective evaluation of external and/or internal demands, as well as environmental stimuli in a given situation. If these stimuli are identified as stressors, they are then subconsciously assessed and classified into various types based on their nature and impact (e.g., as a threat, challenge, harm, or benefit). Secondary appraisal refers to an individual’s evaluation of their available personal resources to cope with the stressor. Together, the appraisal of the stimulus and the perceived adequacy of coping resources determine the emotional, physiological, and behavioral outcomes. For instance, perceiving a stressor as a challenge (potentially beneficial for future performance or well-being) and believing one has sufficient resources to cope tends to evoke positive emotions. Conversely, perceiving a stressor as a threat with inadequate coping resources typically leads to negative emotions such as anxiety (Lazarus, 2000). Anxiety represents a generalized response to an unknown threat, uncertainty, or conflict and encompasses both somatic and cognitive components contributing to the overall experience (Schwartz et al., 1978; Steimer, 2002). Increased anxiety manifests in elevated heart rate, perspiration, and respiration (somatic components), negative thoughts (cognitive components), and behavioral responses such as fight-or-flight tendencies (Steimer, 2002). Psychologically (through emotions) and cognitively (through mental states), anxiety may lead to avoidance behaviors (Robinson et al., 2013), and research has indicated a potential negative relationship between anxiety and working memory (Lukasik et al., 2019). Although researchers frequently decompose this complex model to focus on specific sub processes, Lazarus and others emphasize the inseparable nature of emotions within the stress process. This holistic perspective is particularly relevant in competitive sports, where stressful experiences and associated negative emotions, such as anxiety or frustration, are tightly interconnected (Osório et al., 2017; Tanguy et al., 2018). Recognizing this interplay is essential for understanding how stress impacts referees’ performance and coping mechanisms. Various strategies can be employed to mitigate stress reactions. According to Lazarus’ (1999) Transactional Model of Stress, two general coping strategies are commonly utilized: problem-focused coping and emotion-focused coping. Problem-focused coping involves modifying the subjective experience of a situation through consultation, communication, or increased focus to reduce stress. In contrast, emotion-focused coping aims to manage emotional responses in situations that cannot be changed or influenced, such as audience behaviors or reactions. Drawing from Lazarus and Folkman’s (1984) Transactional Model of Stress, stress management interventions can be structured around four theoretical approaches: 1) reducing the number of potential stressors experienced, 2) modifying cognitive appraisals, including both primary and secondary evaluations, 3) decreasing negative affect while simultaneously boosting positive affect, and 4) strengthening coping strategies to better manage stressors. These approaches collectively aim to mitigate the stress response and enhance overall well-being (Semmer and Zapf, 2018).
Stress, anxiety, and heart rate variabilityStress and negative emotional reactions, such as anxiety, can trigger specific psycho-physiological responses, including perspiration, negative thoughts, and avoidance behavior. These responses can be measured directly or indirectly through tools such as heart rate (HR), heart rate variability (HRV), and validated questionnaires (Ayuso-Moreno et al., 2020; Berntson and Cacioppo, 2007, Spielberger, 1989, Tanguy et al, 2018). While HR measures the number of heart beats per minute, HRV measures the variance of heart beats. In an ECG, this is the time that passes between repeating sequences (usually the R-peak in the QRS-complex). The highest variability in a healthy individual is measured at rest, in which the time variability between individual heart beats is greatest (Berntson and Cacioppo, 2007). In contrast, positive emotional responses to stressors perceived as challenging rather than harmful or threatening -such as excitement, eagerness, and gratification- are associated with different physiological and psychological states (Lazarus and Folkman, 1984). Furthermore, nonverbal behaviors, such as facial expressions and gestures, can provide insight into an individual’s mental state, which may be observable by others (Ashby and Clifton Jr, 2005; Furley et al., 2021; Furley and Schweizer, 2014). Coping with observable stress and anxiety responses requires increased effort and adversely affects attention, concentration, and subsequent performance (Giessing et al., 2019). Research has indicated that both heart rate (HR) and heart rate variability (HRV) exhibit similar reactions under stress and fear conditions (Berntson and Cacioppo, 2007). During periods of heightened physiological and/or psychological stress, heartbeats become more regular as the sympathetic and parasympathetic nervous systems, which normally work antagonistically, instead strive for a synergistic response (Shaffer et al., 2014). This results in decreased HRV compared to resting states (Hottenrott, 2007). Strong experiences of sport-related stress and anxiety are typically associated with reduced HRV and increased HR, which can also be influenced by physical activity levels (Ayuso-Moreno et al., 2020; Fuller, 1992; Mateo et al., 2012; Morales et al., 2013). Stressful conditions, whether physical or psychological, generally increase HR and decrease the variability between each heartbeat, further reducing HRV compared to resting conditions (Fuller, 1992; Hottenrott, 2007). These responses are influenced by factors such as physical fitness, genetics, environment, and other variables (Sammito and Böckelmann, 2016). So far, research on stress and anxiety among referees has predominantly focused on sports such as soccer (Ayuso-Moreno et al., 2020; Johansen and Haugen, 2013; Voight, 2009) or basketball (Burke et al., 2000; Ritchie et al., 2017). In these sports, officials are required to make three types of decisions: (1) uncontroversial calls, (2) subjective judgments regarding technique or misbehavior, and (3) tactical decisions, such as applying sanctions (MacMahon et al., 2014). Decisions of type (2) and (3) are particularly likely to induce stress and anxiety (MacMahon et al., 2014). Soccer and basketball referees must maintain good physical condition to effectively handle tackles, fouls, and positioning demands. In contrast, volleyball refereeing does not require the same level of physical capability, nor does it involve evaluating physical contact between athletes. The key skills for volleyball referees are more cognitive in nature, emphasizing concentration, focus, and rapid decision making due to the fast-paced and brief nature of rallies, the need for technically sound ball handling (no catching or throwing), and the emotionally charged atmosphere of the sport. Previous studies have identified common stressors for volleyball officials, such as the fear of failure or making critical decisions, especially when under evaluation and supervision (e.g., Goldsmith and Williams, 1992; Mirjamali et al., 2012; Rückel et al., 2021). However, while stress reduction and coping strategies have been successful for athletes, no intervention studies have specifically targeted volleyball referees. This study aimed to address this gap by seeking to reduce stress experiences and provide referees with coping strategies to maintain high levels of performance focus and behavior in volleyball officiating (Lagos et al., 2008; Prapavessis et al., 1992; Rumbold et al., 2012).
Study goalThe aims of this study were to identify stressors encountered during volleyball matches, examine their physiological and emotional effects on referees, and evaluate whether a sport psychological intervention aimed at stress management supports referees in coping with these stressors and maintaining their performance. Building on previous research (Ayuso-Moreno et al., 2020; Kostrna and Tenenbaum, 2021; Lagos et al., 2008; Mateo et al., 2012; Morales et al., 2013; Prapavessis et al., 1992; Rumbold et al., 2012), we formulated the following hypotheses:
Hypothesis 1: Stress and anxiety among referees increase HR and decrease HRV.
Hypothesis 2: A sport psychology intervention aids referees in coping with psychological stress, thereby reducing stress and anxiety responses and facilitating sustained performance.
The entire project adopted a mixed-methods-approach (e.g., Sparkes, 2015), integrating both quantitative and qualitative methodologies. Some qualitative findings have previously been reported by Rückel et al. (2021), who conducted and analyzed interviews with referees. In these interviews, referees provided insights into their match perceptions, experienced stressors, stress responses, and coping strategies. They also highlighted the need for strategies to better regulate their emotions, especially in critical and emotionally challenging situations. Building on this, the current study focused on identifying stressors and quantitatively analyzing stress responses in volleyball referees, using instruments such as the BOSSS-d. The referees included in this study were the same participants as those featured in Rückel et al. (2021), and the data presented here were collected concurrently with the data for the prior publication. However, these quantitative data have not been published yet. Ethics approval for the current study was obtained from the ethics committee of the lead university (Ethics Committee Approval Number: 013/19), and data collection took place from January 2019 to December 2020.
Participants and volleyball matchesA total of 38 officials participated in the study (24 males, 14 females, mean age M = 38.29 years, SD = 7.91 years). Although we did not conduct an a priori power analysis, our goal was to recruit as many elite referees as possible, given the limited number of elite volleyball referees in Germany who work in the highest leagues. Twenty participants were randomly assigned to an intervention group, while the remaining 18 were assigned to the control group. The referees were assigned to matches by the German referee manager, who oversees all nominations. The two sport psychologists (one female, one male) selected the game venues and dates based on their availability to conduct the first intervention round. Depending on their assignment to an intervention date and venue, referees were allocated to either the intervention or control group. This process was double-blind, meaning that neither the sport psychologists nor the referee manager were aware of the allocation prior to the referees being assigned to the matches. This method ensured comparable sample sizes between the two groups, despite two participants dropping out at the beginning. Given that no known covariate affected this study, block randomization was considered an appropriate measure (Kang et al., 2008). Participants were assessed while officiating a volleyball match in either the 1st (premier) or 2nd German volleyball league, or at the under-18/under-20 male youth championship, on two occasions. The intervention group underwent a four-step stress management program facilitated by two sport psychologists. The control group, as well as all other officials on the Bundesliga referee list, were provided with information about the techniques, content, and outcomes of the intervention after the project’s completion. At the time of data collection, the sample had been active in the 2nd Bundesliga for an average of 9 years (M = 8.58, SD = 4.33, N = 38). Eighteen participants also held the highest national officiating license, permitting them to referee first division matches (1st Bundesliga). Those with this highest officiating license had been active in first division matches for about 8 years (M = 7.94, SD = 6.73, N = 18). During a season, referees officiated an average of 26 matches (M = 26.36, SD = 6.46). Due to dropouts and limitations caused by the Covid-19 pandemic, complete data sets for the pre-post-comparison were available from 21 referees, with 13 in the intervention group and 8 in the control group. The overall attrition rate of this study is 43% due to the impact of Covid-19, with a differential attrition rate between the groups of 9% in favor of the intervention group.
Materials and apparatusThree questionnaires were employed in this research. First, the trait and state versions of the State-Trait-Anxiety-Inventory (STAI) by Grimm (2009) were administered before and after the match to measure anxiety. Each version consists of ten items rated on an 8-point Likert scale, ranging from 1 (“almost never”) to 8 (“almost always”). The STAI is a reliable and valid questionnaire, with internal consistency coefficients from .86 to .95 (Spielberger et al., 1983). Test-retest reliability coefficients range from .65 to .75 over a two-month interval (Spielberger et al., 1983). Additionally, the STAI demonstrates high construct and concurrent validity, with correlations to Manifest Anxiety Scale ranging from .73 to .90 (Spielberger et al., 1983). Second, the Basketball Officials Source of Stress Survey (BOSSS-d) by Brand (2001) was utilized to assess reported sources of stress during officiating. This survey was slightly modified to better align with volleyball officiating, specifically item 12, which was rephrased from “Calling a technical foul” to “Calling a technical error”. The reliability and validity of the Source of Stress Survey have been well established, with internal consistency of r = 0.91 and internal reliability of r = 0.93 (Mirjamali et al., 2012). The items describe typical match situations and are rated for their stress-inducing effect on a 10-point Likert scale (1 = “This situation left me completely unimpressed”; 10 = “This situation impacted me dramatically”). Lastly, the Inventory of Quality Sport Psychological Support (QS17) by Kleinert and Ohlert (2012) was implemented to evaluate the intervention program. Items such as “I learned new sport psychological techniques” were rated on a 4-point Likert scale ranging from 0 (“Does not apply at all”) to 3 (“Fully applies”). The QS17 has been proven to be highly reliable, with internal consistency values (Cronbach’s alpha) ranging from .74 to .83 (Kleinert and Ohlert, 2014). It includes two versions: one for evaluating a specific sport psychological session and another for overall satisfaction with the support. For this study, the latter version was used. Physiological responses associated with stress and emotional states were objectively measured using HR and HRV. Chest straps from the brand “acentas GmbH” were used for this purpose. A two-minute baseline measurement of HR and HRV was conducted before the match. Subsequently, participants wore the chest straps while officiating to capture variations in HR and HRV. This methodology allowed for the comparison of physiological and psychological stress across referees, both before and after the intervention.
ProcedureData collection before matches took approximately 25 minutes and was conducted in the referees’ locker rooms. Upon arrival, participants signed the informed consent form and completed a demographic questionnaire that included questions on age, gender, referee experience, and motivation to participate in the study. Following this, they filled out both versions of the STAI questionnaires. For baseline HR and HRV measurements, participants were instructed to sit on a bench with their legs at a 90° angle, hands resting on their laps, and eyes closed. During the match, HR and HRV were continuously recorded, beginning with the official introduction of the referees and the competing teams and continuing until the referees left the field at the end of the match. Data collection was conducted using a chest strap device, which measured HR and HRV at rest, both pre-match and post-match, as well as throughout the entire duration of the match. Post-match, approximately 15 minutes after completing their duties, referees filled out the BOSSS-d and the state version of the STAI. Subsequently, a second measurement of resting HR and HRV (recovery measurement) was conducted. The intervention group additionally completed the QS17 following the intervention program. All data were collected and analyzed anonymously. The complete study design is illustrated in Figure 1.
Intervention programThe intervention program applied was based on Lazarus and Folkman’s (1984) Transactional Model of Stress, which provides a rationale for explaining intra-individual and inter-individual differences in stress reactions. The stress management training consisted of four sessions over several months. The intervention included an approach that: 1) focused on trust and relationship building, 2) employed psycho-educational components, 3) used mindfulness-based techniques, 4) implemented tools for cognitive re-appraisal and resource activation, 5) incorporated embodiment exercises, and 6) used systemic reflection to consolidate key insights and lessons learned (Supplement Table S1). Participating in all four sessions was a prerequisite for inclusion in the intervention group. In total, two sessions of online coaching and two in-person group workshops were conducted. For individual meetings, participants were randomly assigned to one of two sport psychology experts certified by the German Federal Institute of Sport Science; the workshops were conducted by both experts, depending on their availability.
Data analysisThe individual questionnaires were prepared and re-coded to derive values for total scores and sub-scales (Brand, 2001; Grimm, 2009; Kleinert and Ohlert, 2012). Cases with missing items were excluded from the analyses using pairwise deletion. For the analyses of the physiological parameters, the programs Kubios v2.2 and HRM Team Monitoring (acentas GmbH) were employed. The processing of the data varied based on the timing of the values: 1) before the match, 2) during the match, and 3) after the match. General data processing involved converting the individual measurements to Kubios, deleting artifacts, determining the two-minute intervals for the pre- and post-match measurements, and calculating HR and HRV (root mean square of successive differences – RMSSD) for the entirety of the respective match. The data was tested for normal distribution, and an additional analysis was conducted to examine gender differences. Gender was included in the analysis due to previous reports indicating significant differences in anxiety and reported stress between men and women (McLean and Anderson, 2009). The baseline data (N = 38) and the pre-post comparison data (N = 21) did not meet the requirements of normal distribution (Shapiro-Wilk test p < 0.05). Therefore, gender differences were analyzed using the Mann-Whitney-U test, which revealed significant differences for HRV post-match (z = -2.53, p = 0.01, r = 0.421, MRank(Male) = 14.95; MRank(Female) = 24.07), problematic single decision (z = -2.52, p = 0.01, r = 0.420, MRank(Male) = 15.22; MRank(Female) = 24.31), fear of physical harm (z = -2.58, p = 0.01, r = 0.442, MRank(Male) = 15.35; MRank(Female) = 22.00), and BOSSS-d total (z = -2.02, p = 0.04, r = 0.368, MRank(Male) = 13.38; MRank(Female) = 20.44). Consequently, gender was included as a covariate in the following analyses. A partial Pearson correlation was applied to test our first hypothesis. Despite the violation of normal distributed data, research has shown a high robustness of this statistical test for non-normal data (Havlicek and Peterson, 1976). For our second hypothesis, we conducted three mixed-model MANCOVAs to assess the effectiveness of our intervention. The MANCOVAs were computed separately for all cardiac responses, the subscales of the STAI and the BOSSS-d questionnaires (Bonferroni correction was applied). Gender and group were treated as between-subject factors, while time and outcome variables were considered within-subject factors. The assumption of a normal distribution was not met for some questionnaires or physiological parameters. However, simulation studies have proven that repeated measures analyses of variance are mostly robust to deviations from a normal distribution (Berkovits et al., 2000; Vasey and Thayer, 1987). Nonetheless, we applied a non-parametric Friedman test in order to compare our results to a non-parametric equivalent. This approach did not allow the inclusion of the between-subject factors gender and group. Consequently, the results of the non-parametric equivalent should be interpreted cautiously, as the reason for possible differences could not be identified. Despite this limitation, it served as a control for the applied RM-MANCOVA and was thus integrated into this research. The Intraclass Correlation Coefficients (ICCs) for the questionnaires used in the study were calculated to assess the reliability of the measurements. Interpretations of the ICC were selected according to Koo and Li (2016). The ICCs were determined via a One-Way Random model and the consistency measurement. The ICCs for the STAI (first measure) were 0.798; 95% CI[0.685, 0.882], indicating a good reliability. The ICCs for The STAI (second measure) were 0.793; 95% CI[0.632, 0.903], indicating a good reliability. For the BOSSS-d, the ICCs were 0.886; 95% CI[0.816, 0.938] indicating a good reliability. The QS17 showed ICCs of 0.801; 95% CI[0.591, 0.932], indicating a good reliability.
The results from the baseline measurements (n = 38) were considered to answer the first hypothesis because baseline data provide a controlled starting point for comparisons. While ongoing measurements during the match were collected, these were excluded from the primary analysis of hypothesis 1 due to potential confounding influences (e.g., external factors like match dynamics, referee decisions, or interactions with players). Instead, the ongoing measurements were utilized for exploratory analyses to better understand stress responses over time. To observe the efficacy of the intervention (hypothesis 2), the full data set of participants who completed both measuring points (n = 21) was used. Additionally, to assess content and process quality of the intervention program, data from those 14 participants in the intervention group (N = 20) who completed the QS 17 was used.
Hypothesis 1In our first hypothesis, we assumed a relationship between anxiety, stress, HR, and HRV for referees officiating a volleyball match. As the respective tables show, significant relationships were found between cardiac responses, stress, and anxiety. Supporting our assumptions, a negative relationship of HRV and stress was found (HRV pre-match and stress with problematic single decisions r = -0.36, p = 0.04). Higher HRV values, reflecting a greater state of relaxation before the match, were associated with fewer reported sources of stress during challenging decision-making scenarios. We also assumed a positive relationship between stress and anxiety, such that referees reporting heightened stress levels would also report higher anxiety. This assumption was supported for state anxiety post-match and stress with problematic single decisions (r = 0.34, p = 0.05) and technical errors (r = 0.41, p = 0.02). Contrary to our expectations, a negative relationship between anxiety before the match and stress after decisions involving technical errors was found (r = -0.37, p = 0.04, see Table 2, Table 4, and Table 6). Furthermore, significant relationships were revealed within the subscales of the questionnaires and cardiac measurement (Table 1, Table 3 and Table 5). As shown in Table 1, the results indicated an elevation in HR during the match compared to the resting measurements, which remained consistent. This could be explained by the increased activity and arousal experienced during the match compared to the rest condition. HRV was reduced significantly during the match compared to the resting situation. Post-hoc analysis revealed a test specific power of 1-β = 0.45 (N = 38, r = 0.30). However, as indicated in Table 3, officials, in general, reported minimal anxiety when officiating a match. The referees documented slightly higher anxiety levels pre-match compared to post-match. Low values were reported for the trait-questionnaire as well. Additionally, referees reported low levels of sources of stress (Table 5). It is important to note that, due to the high number of correlations calculated, some of the results obtained for our hypotheses may be attributed to type-1 errors. Using a corrective analysis like Bonferroni would be overly conservative for effective result analysis, increasing the likelihood of committing a type-2 error. The results obtained from the correlation analysis can be broadly classified into three groups: the HRV statistic group, the BOSSS questionnaire group and intermediate correlations between the two. As it is not surprising that intra-measure correlations are significant, we mostly focus on the significant correlations between the two measures.
Hypothesis 2Via RM-MANCOVA differences between pre- and post-intervention measurements were analyzed. The means and standard deviations of the questionnaires and cardiac reactions are depicted in Table 7. The RM-MANCOVA did not reveal significant multivariate effects for time, F(10,1) = 39.703, p = 0.123, ηp2 = 0.143. However, the multivariate time x group interaction was significant, F(10,1) = 263.148, p = 0.048, ηp2 = 0.215. When comparing the results from the pre-measurements and post-measurements, a slight reduction in post-match state-anxiety can be observed for both groups whereas pre-match state anxiety and trait anxiety remained unchanged. Overall, neither differences over time, F(1,13) = 1.79, p = 0.20, ηp2 = 0.12, nor a time x group interaction effect were significant, F(1,13) = 0.22, p = 0.65, ηp2 = 0.02. A post-hoc analysis showed a test specific power of 1-β = 0.15. Similar results were obtained for the stress questionnaire and cardiac reactions. Referees within the intervention group rated match-specific stress slightly higher at the post-measurement for problematic single decisions (4.92 compared to 2.80), technical errors (2.83 compared to 2.77), social conflicts (3.67 compared to 1.48), as well as overall stress (3.21 compared to 2.09). Despite this increase, the results still reflect only low levels of experienced stress. Looking at the univariate tests, no significant time effects were found; however, certain time x group interactions were observed, including problematic single decision, F(1,17) = 4.581, p = 0.047, ηp2 = 0.212, and social conflicts, F(1,17) = 4.701, p = 0.045, ηp2 = 0.217. For problematic decisions, the intervention group increased their value (from M = 2.31, SD = 2.73 to M = 5.12, SD = 3.6) compared to the reduction of the control group (from M = 3.87, SD = 2.92 to M = 2.84, SD = 1.65). In terms of HR, the only effect that proved to be significantly different in the cardiac measurements between the groups is an interaction effect for the mean heart rate, F(1,15) = 7.914, p = 0.013, ηp2 = 0.345. Other (non-significant) trends were found in post-measurement heart rate interaction effects, F(1,17) = 4.124, p = 0.058, ηp2 = 0.195, and pre-measurement HRV interaction effects, F(1,17) = 3.705, p = 0.071, ηp2 = 0.179. All analyses were controlled for gender. Looking at the means, the mean pulse during the match decreased for the control group, while the mean pulse of the intervention group slightly increased (Control Group: M = 111.57, SD = 13.16 to M = 102.20, SD = 13.73; Intervention Group: M = 101.93, SD = 13.60 to M = 108.48, SD = 10.94). The post-match HR of the intervention group was slightly higher at the post-measurement compared to the baseline (Control Group: M = 83.19, SD = 8.95 to M = 80.13, SD = 11.30; Intervention Group: M = 78.95, SD = 8.52 to M = 84.10, SD = 9.87). Furthermore, a decrease in pre-match HRV was observed for the intervention group, and an increase in pre-match HRV for the control group (Control Group: M = 25.14, SD = 14.77 to M = 38.04, SD = 23.16; Intervention Group: M = 31.02, SD = 22.54 to M = 23.43, SD = 19.58). During the match, an increase in HRV was observed for both groups, which is indicative of an elevated state of relaxation (Control Group: M = 15.72, SD = 9.43 to M = 27.04, SD = 17.92; Intervention Group: M = 17.18, SD = 9.41 to M = 22.81, SD = 14.96). An additional non-parametric Friedman test was conducted due to the attainment of relatively low values for test-specific power. For the STAI, significant differences between the scales of the baseline and post-measurement were found, χ2(5) = 11.21, p < 0.05, n = 17. However, pairwise comparisons remained non-significant. As for the BOSSS-d and the cardiac reactions, no differences within the repeated estimates were found (BOSSS-d overall χ2(11) = 51.05, p < 0.001, n = 16, cardiac reactions overall χ2(11) = 134.80, p < 0.001, n = 15).
Evaluation of the sport psychological intervention programOut of the 20 referees that completed the intervention, 14 answered the QS17. This was due to complications arising from the Covid-19 pandemic. Results of the QS17 are presented in Table 8. The questionnaire was distributed to the intervention group during the post-measurement or in an online format when post-measurements had to be cancelled. The QS17 assisted in assessing the satisfaction and effectiveness of the stress management intervention program for the referees. Evaluating the results against the re-ference values for athletes presented by Kleinert and Ohlert (2012), the intervention program was deemed highly successful. Our values for all three subscales surpass the corresponding norm values established for the existing group. Participants in the intervention program rated the coaching relationship as beneficial and supportive. Furthermore, the referees reported an increased sense of self-efficacy and agency during competitions and in daily life, attributable to the acquired skills and techniques. More specifically, especially the sub-category “Coaching Relationship” reflects the high quality of the intervention process perceived by participants. Very high scores were recorded for the items “I felt understood” and “The sport psychology consultant was someone I could trust”. Additionally, the item “I could understand concepts and content discussed easily” scored very high. The one item with the lowest score in this sub-category was “The sport psychology consultant raised my interest in sport psychology”, which was not necessarily the main focus of the program intervention. The sub-scale “Skills and Capabilities” also showed very high ratings, e.g., for items like “I realized what I can actually achieve by using my mental game”. A relatively low score was recorded for the item “I now have a better understanding about myself”. Regarding the sub-scale “Performance/Performance State”, high scores for the items “Things learned during the intervention help me improve my performance” and “Things learned during the intervention help me in situations outside of sport” indicate the positive impact the program had on participants’ performance. They were able to transfer mental skills and interventions learned to enhance their performance in both sporting and non-sporting contexts.
The current study examined the relationship between stress, HR, and HRV in elite German volleyball referees. Anxiety measurements, which respond to unknown threats, uncertainty, or conflicts, were also included in the analysis. We implemented a randomized controlled trial to assess the effectiveness of a stress management intervention program. The results indicated that stress following a challenging decision or a technical fault was associated with heightened anxiety post-match for the entire sample. This could be attributed to a re-evaluation and re-experiencing of the situation, during which the officials consciously and critically reflected on their performance and decision making. Therefore, our findings complement existing sport psychological research that has demonstrated a significant relationship between stress and anxiety in athletes (Osório et al., 2017; Tanguy et al., 2018) by revealing similar mechanisms in high-level referees. We also identified a significant relationship between HRV pre-match and stress related to problematic single decisions, indicating that higher relaxation and increased HRV values before matches were associated with lessened stress experiences, and vice versa. Generally, higher HR and lower HRV were measured during the match, indicating increased psychophysical stress (Lazarus and Folkman, 1984). Giving the physical requirements of volleyball referees, physical stress is minimal. The study population reported generally low sources of anxiety and stress, aligning with previous findings that referees typically experience only mild anxiety or stress during officiating (Burke et al., 2000; Johansen and Haugen, 2013). This phenomenon can be attributed to the unique dynamics of the volleyball environment and the use of individual coping strategies (Lazarus and Folkman, 1984). Rückel et al. (2021) identified six coping strategies employed by referees, including self-regulation, enhancing focus, and cognitive restructuring, among others. Their study revealed a significant relationship between cognitive stress reactions (e.g., overthinking or difficulty maintaining focus) and emotional responses (e.g., anxiety or frustration) during stressful game situations. These findings underscore the importance of regulating thoughts and emotions in line with the Transactional Theory of Stress. Recent research suggests that applying game management strategies can help referees maintain consistency and control in their decision making, thereby reducing stress (Schrödter and Klatt, 2023). The sport psychological intervention program aimed to reduce stress and anxiety experiences for volleyball referees, integrating the following goals into our stress management intervention: 1) reduction of stress, 2) adaptation of cognitive appraisals, 3) decrease of negative affect, and 4) enhancement of coping strategies. We evaluated the effectiveness of the program through a comparison of pre- and post-intervention measurements. Our analysis revealed no significant differences between pre- and post-intervention measurements in anxiety, officiating-related stress, or cardiac responses. Consequently, our study did not confirm the effectiveness of the stress management intervention for goals one and three for elite volleyball referees. However, notably, the high QS17 ratings reported for items/subscale “Performance/Performance State” specifically indicate a perceived benefit of the stress management program for individuals in the intervention group. Specifically, the high ratings for items such as “I realized what I can actually achieve by using my mental game”, “Things learned during the intervention help me improve my performance”, and “Things learned during the intervention help me feel better during a competition” can be linked to goals 2 and 4 of the stress management intervention: an adaptation of cognitive evaluations (goal 2), and enhanced coping strategies (goal 4). This aligns with findings from previous cognitive anticipation research, which suggest that sport officials benefit from structured cognitive training to enhance decision-making accuracy and emotional regulation (Schrödter et al., 2023). The literature shows that stress management interventions can optimize stress levels in competitive athletes, though the combined effect on stress and performance remain relatively weak (Rumbold et al., 2012). Lagos et al. (2008) and Prapavessis et al. (1992) reported positive effects of stress and anxiety reduction interventions in sport contexts, particularly on competitive anxiety. The generally infrequently reported experiences of stress and anxiety in volleyball referees might explain the limited effects of the intervention. Moreover, the relatively low score on the statement “I now have a better understanding about myself” could indicate a ceiling effect. Many participants, being high-performing elite referees and successful professionals outside of volleyball, likely already possessed a relatively high level of self-awareness before entering the intervention program. This intervention was designed to support a broad range of professional referees. However, it would have been advantageous to conduct additional pre-screening to identify referees experiencing significant demands and strain, enabling a more tailored intervention for those individuals. Future research should examine whether stronger effects emerge among referees experiencing higher levels of stress and anxiety, particularly through tailored volleyball-specific interventions.
Theoretical and practical implicationsOn a theoretical level, this research contributes to the existing literature on stress in high-performance sports. Traditionally, stress research in this domain has primarily focused on athletes; however, the current study broadens this perspective by investigating referees as significant stakeholders within a stress-management framework. This expands our understanding of stress responses in sports beyond athletes to include officials, highlighting their unique stressors and providing insights for future inquiries into referees and their perceptions of stressors. Subsequent research could delve deeper into stress management interventions tailored specifically for highly stressed referees, exploring coping strategies and their impact on performance and well-being. In terms of practical relevance, the findings from this study, along with those from Rückel et al. (2021), have influenced the integration of psychological aspects of refereeing into education and training programs in Germany. There is a growing emphasis on enhancing referees’ performance through personalized coaching for individuals or small groups, acknowledging the psychological challenges they face and equipping them with tools to manage stress effectively. This dual theoretical and practical contribution underscores the importance of addressing psychological factors in officiating, ultimately aiming to optimize performance and well-being among sports officials.
Strengths & LimitationsThe study’s design and intervention program were developed based on existing literature (e.g., Burke et al., 2000; Johansen and Haugen, 2013; Voight, 2009) as well as a specific theoretical framework and model (Lazarus, 2000). Additionally, the active involvement of two sport psychology experts in the study’s development contributed to its external validity, ensuring applicability and transferability in sporting contexts. Their collaboration ensured meticulous methodological planning and thorough interpretation of results, thereby fortifying the robustness and credibility of our findings. However, our hypotheses were only partially confirmed by this research, likely due to notable structural or situational limitations. First, the sample size was nearly halved due to the premature end of the 2019/20 volleyball season as a consequence of the Covid-19 pandemic. The differential attrition rate between our groups was only 9%. While we cannot completely rule out the possibility, it seems highly unlikely that any of the intervention programs led to different rates of attrition. Thus, due to the small group size and external factors that impacted the study results (matches with and without spectators as well as drop-outs of participants due to Covid-19) the advantages of adopting a randomized-controlled trials design were not as evident in our results. Second, stress was measured as an outcome variable twice, whereas it can actually better be understood as a process variable. We tried to integrate this understanding by including physiological parameters, however, further measurements or data collection points might be necessary to gain better insight into the development of individual stress management.
This research represents the first endeavor to analyze stress, anxiety, and cardiac responses among elite volleyball referees during officiating. Additionally, we implemented a theory-based intervention program aimed at assisting referees in coping with stress and anxiety. Our findings indicate that volleyball referees generally report only marginal match-related sources of stress and anxiety, which appear to be unrelated to HR and HRV. No statistically significant effect was found in the pre-post comparison between the intervention and control groups, indicating that the effectiveness of the intervention program could not be scientifically proven. Despite these outcomes, referees assessed the sport psychological intervention as beneficial and helpful, expressing a need for further support. Based on this feedback and related work of Rückel et al. (2021), there appears to be a high demand for stress management interventions not only among athletes and coaches but also among referees. While this assumption awaits validation through additional future research, we highlight this area as a promising field for both research and applied sport psychology practice.
ACKNOWLEDGEMENTS |
This study has been funded by the German Federal Institute for Sport Science (Bundesinstitut für Sportwissenschaft BISp, ZMVI4-071004/19-20). We would like to express our gratitude to the German Volleyball Federation and the German Volleyball League for their cooperation. The authors report no actual or potential conflicts of interest. While the datasets generated and analyzed in this study are not publicly available, they can be obtained from the corresponding author upon reasonable request. All experimental procedures were conducted in compliance with the relevant legal and ethical standards of the country where the study was carried out. |
|
AUTHOR BIOGRAPHY |
 |
Stefanie Klatt |
Employment: Professor at the German Sport University Cologne and Head of the Section Cognition in Team Sports |
Degree: Ph.D. |
Research interests: Investigation of physical, perceptual-cognitive, and adaptive processes in exercise training and their application in (team) sports |
E-mail: s.klatt@dshs-koeln.de |
|
 |
Lisa-Marie Bea |
Employment: Graduate in Sport Psychology |
Degree: B.Sc. |
Research interests: Stress and stress management interventions for volleyball referees |
E-mail: lisa-marie.rueckel@posteo.de |
|
 |
Sebastian Brückner |
Employment: Applied Sport Psychology Consultant, affiliated with the University of Münster |
Degree: Ph.D. |
Research interests: Coaching and performance/stress management consulting in applied sports settings |
E-mail: s.brueckner@uni-muenster.de |
|
 |
André Jungen |
Employment: Head of the Corporate Taxation Center at the Department of Business and Economics, TU University Dortmund |
Degree: Ph.D. |
Research interests: Volleyball refereeing (President of the German Refereeing Commission) |
E-mail: andre.jungen@tu-dortmund.de |
|
 |
Benjamin Noël |
Employment: Research Assistant at the German Sport University Cologne |
Degree: Ph.D. |
Research interests: Anxiety, attention, and their impact on decision making in referees and athletes |
E-mail: ben.noel@web.de |
|
 |
Bernd Strauss |
Employment: Professor at the University of Münster and Head of the Institute of Sport and Exercise Sciences |
Degree: Ph.D. |
Research interests: Sport psychology |
E-mail: bstrauss@uni-muenster.de |
|
|
|
REFERENCES |
 Ashby J., Clifton C. (2005) The prosodic property of lexical stress affects eye movements during silent reading. Cognition 96, 89-100. Crossref
|
 Ayuso-Moreno R., Fuentes-García J. P., Collado-Mateo D., Villafaina S. (2020) Heart rate variability and pre-competitive anxiety according to the demanding level of the match in female soccer athletes. Physiology & Behavior 222, 112926. Crossref
|
 Baltzell A., Summers J. (2018) . The power of mindfulness: Mindfulness meditation training in sport (MMTS) Springer. Crossref
|
 Beckmann, J. and Ehrlenspiel, F. (2018) Strategien der Stressregulation
im Leistungssport. In R. Fuchs & M. Gerber (Hg.), Handbuch
Stressregulation und Sport. Crossref
|
 Berkovits I., Hancock G. R., Nevitt J. (2000) Bootstrap resampling approaches for repeated measure designs: Relative robustness to sphericity and normality violations. Educational and Psychological Measurement 60, 877-892. Crossref
|
 Berntson, G. G. and Cacioppo, J. T. (2007) Heart rate variability: Stress
and psychiatric conditions. In: Malik M, Camm J, (editors)
Dynamic Electrocardiography: Blackwell Publishing
|
 Brand, R. (2001) Stress und Stressbewältigung bei Schiedsrichtern im
Sport [Dissertation]. Universität Konstanz.
|
 Brückner S. (2019) . Using Loehr*s (1989) “16 Second Cure” model as a framework for Mindfulness, ACT and Emotional Self-Regulation Interventions. Paper presented at the 3rd Summit on Mindfulness and Acceptance Approaches in Elite Sport, Magglingen, Switzerland.
|
 Burke K. L., Joyner A. B., Pim A., Czech D. R. (2000) An exploratory investigation of the perceptions of anxiety among basketball officials before, during, and after the contest. Journal of Sport Behavior 23, 11-19.
|
 Cunningham I., Simmons P., Mascarenhas D. (2018) Sport officials' strategies for managing interactions with players: Face-work on the front-stage. Psychology of Sport and Exercise 39, 154-162. Crossref
|
 Ellis A. (1993) . Grundlagen und Methoden der rational-emotiven Verhaltenstherapie. Pfeiffer.
|
 Ellis S., Carette B., Anseel F., Lievens F. (2014) Systematic reflection: Implications for learning from failures and successes. Current Directions in Psychological Science 23, 67-72. Crossref
|
 Fuller B. F. (1992) The effects of stress-anxiety and coping styles on heart rate variability. International Journal of Psychophysiology 12, 81-86. Crossref
|
 Furley P., Schweizer G. (2014) The expression of victory and loss: Estimating who’s leading or trailing from nonverbal cues in sports. Journal of Nonverbal Behavior 38, 1329. Crossref
|
 Furley P., Schweizer G., Laborde S. (2021) Emotional intelligence and drawing inferences from nonverbal cues in sports. International Journal of Sport and Exercise Psychology 20, 1617-1637. Crossref
|
 Giessing L., Frenkel M. O., Zinner C., Rummel J., Nieuwenhuys A., Kasperk C., Brune M., Engel F., Plessner H. (2019) Effects of coping-related traits and psychophysiological stress responses on police recruits’ shooting behavior in Reality-Based scenarios. Frontiers in Psychology 10. Crossref
|
 Goldsmith P. A., Williams J. M. (1992) Perceived stressors for football and volleyball officials from three rating levels. Journal of Sport Behavior 15, 106.
|
 Grimm, J. (Ed.) (2009) State-Trait-Anxiety Inventory nach Spielberger.
Deutsche Lang- und Kurzversion. MF-Working Paper 2009/02.
|
 Havlicek L. L., Peterson N. L. (1976) Robustness of the Pearson correlation against violations of assumptions. Perceptual and Motor Skills 43, 1319-1334. Crossref
|
 Hottenrott, K. (2007) Training with the heart rate monitor. Meyer &
Meyer Verlag.
|
 Høigaard R., Johansen B. T. (2004) The solution-focused approach in sport psychology. The Sport Psychologist 18, 218-228. Crossref
|
 Johansen B. T., Haugen T. (2013) Anxiety level and decision-making among Norwegian top-class soccer referees. International Journal of Sport and Exercise Psychology 11, 215-226. Crossref
|
 Kang M., Ragan B. G., Park J. (2008) Issues in outcomes research: An overview of randomization techniques for clinical trials. Journal of Athletic Training 43, 215-221. Crossref
|
 Kleinert J., Ohlert J. (2014) Ergebnisqualität in der sportpsychologischen Beratung und Betreuung. Zeitschrift für Sportpsychologie 21, 13-22. Crossref
|
 Kleinert, J. and Ohlert, J. (2012) Erfassung von Betreuungsqualität in der
Sportpsychologie: Das Befragungsinventar QS17. Cologne:
German Sport University Cologne.
|
 Koo T. K., Li M. Y. (2016) A guideline of selecting and reporting intraclass correlation coefficients for Reliability Research. Journal of Chiropractic Medicine 15, 155-163. Crossref
|
 Kostrna J., Tenenbaum G. (2021) Developing and testing the expanded sport official’s decision-making model. International Journal of Sport and Exercise Psychology 20, 586-611. Crossref
|
 Lagos L., Vaschillo E., Vaschillo B., Lehrer P., Bates M., Pandina R. (2008) Heart rate variability biofeedback as a strategy for dealing with competitive anxiety: A case study. Biofeedback 36, 109-115.
|
 Lazarus R. S. (2000) How emotions influence performance in competitive sports. The Sport Psychologist 14, 229-252. Crossref
|
 Lazarus, R. S. (1999) A new synthesis: stress and emotion. New York.
Springer Publishing Company.
|
 Lazarus, R. S. and Folkman, S. (1984) Stress, appraisal, and coping.
Springer publishing company.
|
 Loehr, J. (1989) Mental toughness training for tennis: "The 16 Second
Cure". Video produced by FTM Sports. 14500 S.W. 119 Ave.,
Miami, FL 33186.
|
 Lukasik K. M., Waris O., Soveri A., Lehtonen M., Laine M. (2019) The relationship of anxiety and stress with working memory performance in a large non-depressed sample. Frontiers in Psychology 10. Crossref
|
 MacMahon C., Mascarenhas D., Plessner H., Pizzera A., Oudejans R., Raab M. (2014) Sports officials and officiating: Science and practice. Routledge. Crossref
|
 Mateo M., Blasco-Lafarga C., Martínez-Navarro I., Guzmán J. F., Zabala M. (2012) Heart rate variability and pre-competitive anxiety in BMX discipline. European Journal of Applied Physiology 112, 113-123. Crossref
|
 McLean C. P., Anderson E. R. (2009) Brave men and timid women? A review of the gender differences in fear and anxiety. Clinical Psychology Review 29, 496-505. Crossref
|
 Mirjamali E., Ramzaninezhad R., Rahmaninia F., Reihani M. (2012) A study of stress in international and national referees of soccer, volleyball, basketball and handball in Iran. World Journal of Sport Sciences 6, 347-354.
|
 Morales J., Garcia V., García-Massó X., Salvá P., Escobar R., Buscà B. (2013) The use of heart rate variability in assessing precompetitive stress in high-standard judo athletes. International Journal of Sports Medicine 34, 144-151. Crossref
|
 Osório, J. M. A., Cruz, J. F. A., Sofia, R. M. C. and Peixoto, M. M. (2017)
Threat appraisals, performance anxiety, and associations with
motivation and emotion regulation in sport competition: An
exploratory study with young athletes. In J. F. A. Cruz & R. M.
C. Sofia (Eds.), Psychology of emotions, motivations and
actions. Anger and anxiety: Predictors, coping strategies, and
health effects (pp. 239-267) Nova Science Publishers.
https://psycnet.apa.org/record/2018-06617-009
|
 Pineschi G., Di Pietro A. (2013) Anxiety management through psychophysiological techniques: Relaxation and psyching-up in sport. Journal of Sport Psychology in Action 4, 181190. Crossref
|
 Prapavessis H., Grove J. R., McNair P. J., Cable N. T. (1992) Self-regulation training, state anxiety, and sport performance: A psychophysiological case study. The Sport Psychologist 6, 213-229. Crossref
|
 Ritchie J., Basevitch I., Rodenberg R., Tenenbaum G. (2017) Situation criticality and basketball officials' stress levels. Journal of Sports Sciences 35, 2080-2087. Crossref
|
 Robinson O. J., Vytal K., Cornwell B. R., Grillon C. (2013) The impact of anxiety upon cognition: perspectives from human threat of shock studies. Frontiers in Human Neuroscience 7. Crossref
|
 Rumbold J. L., Fletcher D., Daniels K. (2012) A systematic review of stress management interventions with sport performers. Sport, Exercise, and Performance Psychology 1, 173-193. Crossref
|
 Rückel L.-M., Noël B., Jungen A., Brückner S., Strauss B., Klatt S. (2021) Reflecting on the match: Situational stressors, stress responses, and coping in German elite volleyball referees. Case Studies in Sport and Exercise Psychology 5, 124-134. Crossref
|
 Sammito S., Böckelmann I. (2016) Factors influencing heart rate variability. International Cardiovascular Forum Journal 6, 18-22. Crossref
|
 Schrödter R., Klatt S. (2023) Context matters: An examination of game management in refereeing. International Journal of Sport and Exercise Psychology 22, 1795-1818. Crossref
|
 Schrödter R., Noël B., Klatt S. (2023) Game management by referees to compensate for errors in judgement: A decision flow model. International Journal of Sport and Exercise Psychology. Crossref
|
 Schrödter R., Schwarting A., Fasold F., Schul K., Klatt S. (2023) The relevance of general spatial anticipation skills for basketball referees. Applied Sciences 13, 2991. Crossref
|
 Schwartz G. E., Davidson R. J., Goleman D. J. (1978) Patterning of cognitive and somatic processes in the self-regulation of anxiety: Effects of meditation versus exercise. Psychosomatic Medicine 40, 321-328. Crossref
|
 Semmer, N. K. and Zapf, D. (2018) Theorien der Stressentstehung und -
bewältigung. In R. Fuchs & M. Gerber (Eds.), Handbuch
Stressregulation und Sport. Crossref
|
 Shaffer F., McCraty R., Zerr C. (2014) A healthy heart is not a metronome: an integrative review of the heart’s anatomy and heart rate variability. Frontiers in Psychology 5. Crossref
|
 Sharp L.-A., Hodge K., Danish S. (2015) Ultimately it comes down to the relationship: Experienced consultants’ views of effective sport psychology consulting. The Sport Psychologist 29, 358-370.
|
 Sparkes A. C. (2015) Developing mixed methods research in sport and exercise psychology: Critical reflections on five points of controversy. Psychology of Sport and Exercise 16, 49-59. Crossref
|
 Spielberger, C. D. (1989) State-Trait Anxiety Inventory: Bibliography
(2nd ed.) Palo Alto, CA: Consulting Psychologists Press.
|
 Spielberger, C. D., Gorsuch, R. L., Lushene, R., Vagg, P. R. and Jacobs,
G. A. (1983) Manual for the State-Trait Anxiety Inventory. Palo
Alto, CA: Consulting Psychologists Press.
|
 Steimer T. (2002) The biology of fear- and anxiety-related behaviors. Dialogues in Clinical Neuroscience 4, 231-249. Crossref
|
 Storch, M., Cantieni, B., Hüther, G. and Tschacher, W. (Hg.) (2006)
Embodiment. Die Wechselwirkung von Körper und Psyche
verstehen und nutzen. Huber.
|
 Tanguy G., Sagui E., Fabien Z., Martin-Krumm C., Canini F., Trousselard M. (2018) Anxiety and psycho-physiological stress response to competitive sport exercise. Frontiers in Psychology 9, 1469. Crossref
|
 Tuero C., Tabernero B., Guillén F., Márquez S. (2002) Analysis of the factors affecting the practice of refereeing. Revista Scape 1, 7-16.
|
 Vasey M. W., Thayer J. F. (1987) The continuing problem of false positives in repeated measures ANOVA in psychophysiology: A multivariate solution. Psychophysiology 24, 479-486. Crossref
|
 Voight M. (2009) Sources of stress and coping strategies of US soccer officials. Stress and Health 25, 91-101. Crossref
|
|
|
|
|
|
|